Automatic summary evaluation based on text grammars

In this paper, I describe a method for evaluating automatically generated text summaries. The method is inspired by research in text grammars by Teun Van Dijk. It addresses a text as a complex structure, the elements of which are interconnected both on the level of form and meaning, and the well-formedness of which should be described on both of these levels. The method addresses current problems of summary evaluation methods, especially the problem of quantifying informativity, as well as the problem of objective measurement of well-formedness of text. It is believed that the ideas from this research can contribute to evaluation methods for algorithms transforming complex meaningful entities into other complex meaningful entities (text, hypertext, sound, vision).